Universal Journal of Educational Research 5(1): 45-53, 2017 
DOI: 10.13189/ujer.2017.050106 


http://www.hrpub.org 


A Validation Study of the Culturally 
Responsive Teaching Survey 

Christy M. Rhodes 


College of Education, East Carolina University, Greenville, 27858, North Carolina, United States 


Copyright©2017 by authors, all rights reserved. Authors agree that this article remains permanently open access under the 
terms of the Creative Commons Attribution License 4.0 International License 


Abstract Amidst the ethnic and linguistic diversity in 
adult English language classes, there is heightened 
importance to using culturally responsive teaching practices. 
However, there are limited quantitative examinations of this 
approach in adult learning environments. The purpose of this 
investigation was to describe patterns of culturally 
responsive teaching practices of adult ESOL teachers and to 
establish the psychometric properties of the Culturally 
Responsive Teaching Survey (CRTS), a newly-developed 
self-assessment survey. Based on Ginsberg and 
Wlodkowski’s Motivational Framework for Culturally 
Responsive Teaching [1], this 17-item online survey 
establishes patterns of teaching praxis appropriate for adult 
English language classrooms. Findings revealed a trend of 
regular to frequent use of the majority of the culturally 
responsive teaching practices indicated in the CRTS. In 
addition, analyses demonstrated that the CRTS is a reliable, 
uni-dimensional scale which yielded positive correlations 
with multi-cultural knowledge and teaching skills. Thus, the 
CRTS provides a useful tool for examining the praxis of 
culturally responsive teaching in adult, second-language 
classrooms. These findings will lead to improved 
understanding of how adult educators incorporate culturally 
responsive teaching practices in ethnically and linguistically 
diverse learning environments, in addition to supporting the 
use of this instrument in future research studies. 

Keywords English for Speakers of Other Languages, 
Culturally Relevant Pedagogy, Adult Education 


1. Introduction 

Given the ethnic and linguistic diversity of adult English 
for Speakers of Other Languages (ESOL) students, the 
incorporation of culture and student cultural identities into 
the learning environment has heightened importance [2-4]. 
While highly diverse in age, country of origin, and native 
language [5], adult English language learners all experience 
the acculturation process, simultaneously maintaining ties to 


their native cultural group and developing relationships 
within their adopted cultural group [6]. Culturally responsive 
teaching mediates the challenges of this process by placing 
students’ cultural norms and practices at the center of the 
learning process [7]. In essence, culturally responsive 
educators are proficient at discerning subtle and overt 
differences and developing culturally-sensitive and 
appropriate learning environments [8,9]. However, while 
interest in culturally responsive teaching is high, there are 
limited quantitative examinations of this approach in adult 
learning environments. In general, research has been 
conducted through qualitative methods or assessments of 
teachers’ cultural sensitivity, racial bias or cultural 
competence [10], not teaching practices. Therefore, there is 
limited quantitative data describing how adult ESOL 
teachers incorporate their student’s cultural norms and 
values into their curriculum and instruction. 

The purpose of this investigation was twofold: to describe 
patterns of culturally responsive teaching practices of adult 
ESOL teachers and to establish the psychometric properties 
of the Culturally Responsive Teaching Survey (CRTS), a 
newly-developed self-assessment survey. Specifically, I 
examined: (a) the self-reported frequency of use of 17 
teaching practices, (b) the factorial structure of the CRTS, 
and (c) the construct validity of the CRTS in studies 
conducted with English language teachers of adults. These 
findings will lead to improved understanding of how adult 
educators incorporate culturally responsive teaching 
practices in ethnically and linguistically diverse learning 
environments, in addition to supporting the use of this 
instrument by researchers and practitioners. 

2. Theoretical Framework 

Culturally responsive teaching is an umbrella term which 
encompasses a variety of approaches, such as culturally 
relevant, culturally sensitive, culturally congruent, and 
culturally contextualized pedagogies [7]. This equity-based 
approach places students’ cultures at the core of the learning 
process and utilizes the “cultural knowledge, prior 
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experiences, frames of reference, and performance styles of 
ethnically diverse students” [7 p. 29]. By creating classroom 
norms reflective of the students’ native cultures, and not 
those of mainstream culture, the culturally responsive 
educator mitigates the challenges of overcoming “cultural 
mismatches” between the home and school [11,12]. 
Culturally responsive teaching is distinguished by its 
emphasis on validating, facilitating, liberating, and 
empowering minority students by “cultivating their cultural 
integrity, individual abilities, and academic success” [7 p. 44] 
and is based on the four pillars of “teacher attitude and 
expectations, cultural communication in the classroom, 
culturally diverse context in the curriculum, and culturally 
congruent instructional strategies” [7, p. 44]. 

While much of culturally responsive teaching theory 
refers to K-12 settings, Ginsberg and Wlodkowski developed 
the Motivational Framework for Culturally Responsive 
Teaching [1] for application in adult learning environments. 
They posited that culturally responsive teaching increases 
the intrinsic motivation of students of non-dominant cultural 
groups. This framework describes the norms and practices of 
an environment in which “inquiry, respect, and the 
opportunity for full participation by diverse adults is the 
norm” [14, p. 161] and is based on the integrated use of four 
elements: establishing inclusion, developing attitude, 
enhancing meaning, and engendering competence [1,14]. 
Each element, or criteria, has corresponding norms and 
practices that adult educators can use in creating or 
evaluating their praxis. 

Teaching practices that create an environment of respect 
and connectedness and that use cooperation and equitable 
treatment of all learners reflect the element of establishing 
inclusion. The element, developing attitude , includes norms 
and practices that help students develop a positive attitude 
toward the learning process by building on students’ 
personal experiences and knowledge and by allowing 
learners to make choices throughout the learning process. 
The third element, enhancing meaning, includes norms and 
practices that encourage students to engage in deep reflection 
and critical inquiry, such as role-plays and simulations. The 
final element, engendering competence, are practices that 
show the learner evidence of his or her learning and 
proficiency and the use of assessments that are 
contextualized in the learners’ experiences [1,14]. 

Ginsberg and Wlodkowski’s framework served as the 
operational framework of the CRTS due to its original 
design as “a tool for continual reflection” [1, p. 39] to help 
educators examine their teaching in an effort to improve the 
cultural responsiveness of their practices. In addition, 
although this framework was designed for the higher 
education classroom, it could be adapted to include the 
unique patterns of teaching practice relevant to adult 
language learning environments. 

3. Survey Development 


Development of a survey instrument can be 
accomplished through various methods; however, the 
process entails the following steps: clear identification of 
the construct to be measured, item generation, expert review 
and refinement of item pool, a pilot or development study, 
and item evaluation and reduction [13]. A list of culturally 
responsive teaching characteristics compiled during a 
literature review served as the theoretical foundation of the 
item development process. A literature review using the key 
terms “culturally responsive”, “culturally relevant”, and 
“culturally congruent” teaching yielded findings from 
predominantly qualitative examinations of culturally 
responsive teachers in a variety of teaching environments. 
All characteristics were compiled into a master list and 
categorized by the four elements of the Motivational 
Framework for Culturally Responsive Teaching [1], The 
list was then consolidated by eliminating redundant 
characteristics, resulting in a master list of 23 characteristics. 
These represented general beliefs and experiences and were 
used to develop items of specific teaching practices 
appropriate for an adult English language learning 
environment. This was achieved by a panel of ESOL 
practitioners describing teaching practices that manifested 
each characteristic. The draft item pool contained 27 
teaching practices. 

To assess the content validity of the item pool, the 
expertise of two panels was utilized. The first validation 
panel consisted of individuals with extensive teaching 
experience in the field of adult ESOL and Teaching English 
to Speakers of Other Languages (TESOL) who evaluated 
the items for clarity and relevance to second language 
teaching theory [15]. The second validation panel consisted 
of experts in culturally responsive teaching who evaluated 
the items for relevance to the theories of adult learning and 
culturally responsive pedagogy. Ranked on a five-point 
scale, items with means of 3 or below were deleted, while 2 
items were reworded or combined. At the conclusion of this 
stage, the draft survey included 17 culturally responsive 
teaching practices. The final phase of survey development 
consisted of both cognitive interviewing and a pilot study 
with a representative sample of 100 adult education ESOL 
teachers. To assess the reliability of the pilot survey, 
Cronbach’s Alpha Coefficient was calculated and deemed 
acceptable at .752. Based on these data, all 17 items were 
retained for the final version of the CRTS. 


4. Study 1 

4.1. Sampling and Procedure 

The CRTS was first used in a study of adult education 
ESOL teachers in Florida in late 2012 [16, 17]. The purpose 
of this study was twofold: to develop and validate the CRTS 
and to describe the self-reported use of 17 specific 
culturally responsive teaching practices of adult ESOL 
teachers throughout the state of Florida. The target 
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population consisted of teachers in non-credit, adult 
education ESOL programs. The sampling frame was 
compiled using publicly-accessible faculty directories from 
15 state or community colleges, 2 universities, 8 school 
districts, and Bay Area Regional TESOL (BART). From the 
430 person sampling frame, there were 134 completed 
surveys resulting in a 31.2% response rate. The majority of 
respondents were females (78.38%) from community or 
state colleges (92%). Four follow-up reminder emails were 
sent over a one-month period. Responses were somewhat 
evenly distributed with 46, 49, and 39 responses in 
consecutive weeks, and no further responses following the 
fourth reminder email. To examine patterns of response by 
completion date, ANOVA and Tukey tests were conducted. 
Significant differences were found using a predetermined 
Type I error rate of .05 in three items, but there was no 
discernible pattern to these differences. Therefore, the 
dataset was analyzed without sub-grouping of response 
times. 

4.2. Instrumentation 

The CRTS includes 17 teaching practices about which 
participants report their frequency of use on a 5-point 
frequency scale with levels of: never, rarely, sometimes, 
usually, and always. Participants respond to items such as, “I 
ask students to compare their culture with American culture’ 
and “7 include lessons about the acculturation process” by 
choosing how often they utilize that practice. See Appendix 
A for the complete item pool. 

4.3. Results 

4.3.1. Reliability 

When examining latent traits such as culturally relevant 
teaching, the reporting of the reliability of sample scores 
establishes a level of consistency of these unobservable 
characteristics [18]. The Cronbach Alpha Coefficient value 
of these means was calculated at .781, revealing appropriate 
levels of internal consistency for data analysis. 

4.3.2. Frequency of Use 

Various descriptive analyses were conducted to describe 
the respondents’ frequency of use. The first was an 
examination and representation of the frequency 
distributions of each item, which were analyzed for central 
tendency, dispersion, and shape. A cumulative frequency 
chart was developed in order to examine overall patterns 
within the sample. Additionally, variance, standard deviation, 
skewness and kurtosis of both the overall and item scores 
were compiled and examined. 

Item means ranged from 2.51 to 4.26 with thirteen of the 
seventeen items falling in the moderate to high range of 


3.02 to 4.26 corresponding to the frequency levels of 
sometimes, usually, or always. The remaining four item 
means ranged from 2.5 to 3.0 corresponding to the 
frequency level between rarely and sometimes. 

The most frequently used practice was ‘‘provide rubrics 
and progress reports to students” (M = 4.26; SD = .98), 
followed closely by “ elicit students’ experiences in 
pre-reading and pre-listening activities” (M = 4.24; 
SD = .748). The items, “ ask students to compare their 
culture with American culture ” and “ make an effort to get to 
know students’ families and background”, were also noted 
as being frequently used with means of 4.16 and 4.10, 
respectively. 

Examination of respondents’ item responses reveals 
further details about the two most frequently used teaching 
practices. While three respondents indicated that they never 
“provide rubrics and progress reports to students ”, roughly 
80% of all surveyed teachers indicated that they did so on a 
highly regular basis. A larger percentage of the sample 
(86.6%), however, indicated that they usually or always 
“elicit students’ experiences in pre-reading and pre-listening 
activities ”. Thus, while both item means were high, more 
teachers use students’ experiences than providing rubrics or 
progress reports. A full list of the mean scores and standard 
deviations, can be found in Table 1. 

Among the least frequently used practices were “include 
lessons about anti-immigrant discrimination or bias” 
(M = 2.51; SD = 1.017), and “students work independently, 
selecting their own learning activities” (M = 2.76; 
SD = .860), followed by “ ask for student input when 
planning lessons and activities” (47=2.91; SD = .921) and 
“use student surveys to learn about students’ classroom 
preferences" (M =2.94; SD = 1.102). When these item 
responses were examined, there was greater dispersion of 
the least frequently used culturally responsive teaching 
practices than the most frequently used ones. A greater 
number of teachers indicated the moderate use of these four 
practices with the frequency level sometimes, in addition to 
both the levels of rarely and usually. For example, even 
though 18.7% of the sample indicated they never included 
lessons about anti-immigrant discrimination or bias, 15 
teachers indicated that they usually or always did so 
comprising 11.2 % of the sample. Slightly more than 
one-third of the sample indicated that they never or rarely 
have students work independently or select their own 
learning activities. When the teachers who sometimes 
engage in this practice are added, an overwhelming 
majority of the total sample (86%) did not support the use 
of this culturally responsive teaching practice. A complete 
percentage distribution of the item responses by scale value 
of the four least used teaching practices can be found in 
Table 2. 
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Table 1. Descriptive Statistics of Responses to How Frequently Teachers Used Culturally Responsive Teaching Practices 


Survey Item 




Study 1 

Study 1 

Study 2 

Study 2 




Mean 

SD 

Mean 

SD 

Provide rubrics and progress reports to students 




4.26 

.980 

4.15 

.956 

Elicit students’ experiences in pre-reading and pre-listening activities 



4.24 

.748 

4.20 

.803 

Ask students to compare their culture with American culture 




4.16 

.793 

4.05 

.743 

Make an effort to get to know students’ families and background 



4.10 

.892 

4.07 

.880 

Use mixed-language and mixed-cultural pairings in group work 



3.91 

1.051 

4.28 

.884 

Examine class materials for appropriate images and themes 




3.90 

1.035 

4.32 

.796 

Encourage students to use cross-cultural comparisons when analyzing material 


3.69 

.853 

3.88 

.921 

Spend time outside of class learning about the cultures and languages of students 


3.40 

.989 

3.67 

.963 

Use peer tutors or student-led discussions 




3.30 

.910 

3.79 

.954 

Learn words in students’ native languages 




3.29 

1.068 

3.53 

1.076 

Supplement the curriculum with lessons about international current events 



3.27 

.935 

3.49 

.864 

Include lessons about the acculturation process 




3.25 

.963 

3.25 

1.021 

Encourage students to speak their native language with their children 



3.02 

1.443 

3.31 

1.495 

Use student surveys to learn about students’ classroom preferences 



2.94 

1.102 

3.01 

1.141 

Ask for student input when planning lessons and activities 




2.91 

.921 

3.28 

.914 

Students work independently, selecting their own learning activities 



2.76 

.860 

2.86 

.953 

Include lessons about anti-immigrant discrimination or bias 




2.51 

1.017 

2.84 

1.027 

Notes. A(Study 1) = 134; N (Study 2) =219 








Table 2. Percentage Distribution of Responses to Least Frequently Used Culturally Responsive Teaching Practices by Scale Values 

Item 

Never 

Rarely 

Sometimes 

Usually 

Always 

Include lessons about anti-immigrant 
discrimination or bias 

18.7% 

27.6% 


42.5% 

6.7% 


4.5% 

Students work independently, selecting their 
own learning activities 

6.0% 

29.9% 


50.0% 

10.4% 


3.7% 

Ask for student input when planning lessons 
and activities 

8.2% 

19.4% 


48.5% 

20.9% 


3.0% 

Use student surveys to learn about students’ 
classroom preferences 

10.4% 

22.4% 


39.6% 

17.9% 


9.7% 

Note. N= 134 








Table 3. Factor Pattern Coefficients Based on a Principle Components Analysis for Items Related to Frequency of Use Sub-Scale 

in Study 1 

Rotated Component Matrix' 1 






Component 






i 

2 

3 

4 

5 

Item 1 — Domain A - Establishing Inclusion 



.675 

-.149 

.253 

-.165 

.005 

Item 2 — Domain B - Developing Attitude 



.168 

.234 

.691 

-.214 

.043 

Item 3 - Domain C - Enhancing Meaning 



.711 

.260 

-.049 

.070 

-.328 

Item 4 - Domain A - Establishing Inclusion 



.435 

.332 

.065 

.236 

.090 

Item 5 — Domain A - Establishing Inclusion 



-.026 

.681 

.112 

-.053 

.201 

Item 6 - Domain A - Establishing Inclusion 



.206 

.690 

-.063 

.044 

-.309 

Item 7 - Domain D - Engendering Competence 



.387 

.185 

.434 

.361 

-.152 

Item 8 - Domain B - Developing Attitude 



.021 

-.025 

.740 

.338 

.087 

Item 9 — Domain B - Developing Attitude 



.544 

.118 

-.219 

.346 

.178 

Item 10 - Domain A - Establishing Inclusion 



.096 

.103 

.025 

-.009 

.762 

Item 1 1 - Domain D - Engendering Competence 



.142 

.139 

-.051 

.641 

.239 

Item 12 - Domain A - Establishing Inclusion 



.135 

.585 

.289 

.188 

.364 

Item 13 - Domain A - Establishing Inclusion 



.559 

-.045 

.269 

.162 

.261 

Item 14 - Domain C - Enhancing Meaning 



.608 

.002 

.123 

.150 

.332 

Item 15 - Domain D - Engendering Competence 



.349 

.337 

.305 

.355 

.301 

Item 16 - Domain C - Enhancing Meaning 



.652 

.259 

.046 

-.089 

.009 

Item 17 - Domain D - Engendering Competence 



-.075 

-.106 

.204 

.689 

-.238 
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4.3.3. Exploratory Factor Analysis 

In this study, an exploratory factor analysis (EFA) was 
conducted to ascertain if certain items functioned as a group, 
or factor, of the construct of culturally responsive teaching 
practices [13]. The appropriateness of the data for an EFA 
was examined based on examining the Kaiser-Meyer-Olkin 
(KMO) measure of sampling adequacy and Bartlett’s test of 
sphericity. The KMO was .767, which is considered 
acceptable for EFA [19,20]. Bartlett’s test of sphericity 
produced a statistically significant value, (/2(136) = 473.19, 
p < .05); therefore, the data were deemed acceptable for 
EFA. 

The EFA produced a five-factor solution using a varimax 
rotation and was used for the final solution. The factor 
pattern coefficients revealed a majority of the 17 items with 
factor loadings of .55 or greater and thus, deemed 
significant indicators of their respective factors [21,22]. 
Using this cutoff point to compare the factor structure of the 
items in this study to the four-element theoretical 
framework provided by the Motivational Framework of 
Culturally Responsive Teaching [1], limited similarities 
were found. The factors on the CRTS accounted for 56% of 
the total variance with eigenvalues higher than 1.0 for each 
of the factors. Therefore, the eigenvalues met the criteria for 
Kaiser’s rule [23], but did not meet the criteria for the total 
variance (greater than 60%) that is considered acceptable 
for research in the social sciences [22,24,25]. A complete 
list of the factor pattern coefficients is presented in Table 3. 

In summary, the findings demonstrated a limited 
relationship to the operational framework. Therefore, while 
additional studies with expanded sampling frames may yield 
a different internal structure, these findings suggest a 
uni-dimensional construct to the CRTS, not the multiple 
element of the theoretical foundation, the Motivational 
Framework of Culturally Responsive Teaching [1]. 

5. Study 2 

5.1. Sampling and Procedure 

The CRTS was also used in a nationwide study of adult 
education ESOL teachers from 2014 to 2016. The purpose 
of this study was also to describe the culturally responsive 
teaching practices of the sample, while also examining the 
construct validity of the CRTS through the administration 
of the Multicultural Teaching Competencies Scale (MTCS) 
[26]. The sampling and survey administration followed 
the same procedures as Study 1. The survey was 
administered to three groups, with the results of ANOVA 
and Turkey tests revealing no discernible pattern to 
differences in date of response. Therefore, the dataset was 
analyzed without sub-grouping. There were 219 respondents, 
the majority of whom were female (88.9%), with males 
representing 10.1% of the sample, leaving 1% of 
undisclosed gender. The majority of respondents were from 


community colleges (26.9%) and colleges or universities 
(58.5%), while individuals from school districts comprised 
only 1.6% of the respondents. 

5.2. Instrumentation 

In addition to the CRTS, participants completed the 
MTCS. The MTCS is a self-report questionnaire that 
assesses skills, behaviors, and knowledge of multicultural 
teaching practices and theory [26]. The MTCS is 
composed of 16 items divided into two sub-scales: 
Multicultural Teaching Skills and Multicultural Knowledge. 
Each item is assessed on a six-point Likert-type scale 
ranging from 1 (Strongly Disagree) to 6 (Strongly Agree). 
Examples of the Multicultural Teaching Skills sub-scale are 
1 plan many activities to celebrate diverse cultural practices 
in my classroom and 1 often include examples of the 
experiences and perspectives of racial and ethnic groups 
during my classroom lessons. Examples of the 
Multicultural Knowledge sub-scale are 1 understand the 
various communication styles among different racial and 
ethnic minority students in my classroom and 1 am 
knowledgeable about the particular teaching strategies that 
affirm the racial and ethnic identities of all students. For 
this study, the item I establish strong, supportive 
relationships with racial and ethnic minority parents was 
not included, due to its lack of relevance to the adult 
learning environment. 

5.3. Results 

5.3.1. Reliability 

Similar to Study 1, the Cronbach Alpha Coefficient of the 
CRTS demonstrated a high level of internal reliability 
at .834. Additionally, both Cronbach Alpha Coefficients of 
the MCTS were acceptable at .850 (Teaching sub-scale) 
and .840 (Knowledge sub-scale) [16]. Therefore, scores 
from both instruments were considered appropriate for 
further analysis. 

5.3.2. Frequency of Use 

The data analysis utilized in Study 1 was replicated in 
this study, yielding frequency distributions of each item and 
a cumulative frequency chart to examine overall patterns of 
this sample. The high patterns of use in Study 1 were also 
reported in Study 2. In overall item responses, item means 
ranged from 2.84 to 4.32 with fourteen items falling in the 
moderate range of 3 and above corresponding to the 
frequency category of sometimes, usually and always. The 
remaining three item means ranged from 2.84 to 2.97, 
corresponding to the frequency level between rarely and 
sometimes. 

The most frequently used practices in this study included 
those found in Study 1 (noted above) and the additional 
items “examine class materials for appropriate images and 
themes” (M = 4.32; SD = .796), followed by “ use 
mixed-language and mixed-cultural pairings in group work” 
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(M = 4.28; SD = .88). While these item means had been 
high in Study 1, they now ranked as the top two most 
frequently used items. Further similarities were found in the 
least frequently used items with the same two items “ include 
lessons about anti-immigrant discrimination or bias” 
(M = 2.84; SD = 1.03) and “ students work independently, 
selecting their own learning activities ” ( M = 2.86; SD = .96) 
receiving the lowest reported frequency. A full list of the 
mean scores and standard deviations can be found in Table 1. 

Examination of item responses once again yields further 
details about the patterns of use of the most and least 
frequently used practices. The item most respondents (49%) 
described as always using was " provide rubrics and 
progress reports to students”, followed by ‘‘examine class 
materials for appropriate images and themes” and ‘‘use 
mixed-language and mixed-cultured pairings in group work”, 
43.1% and 42.8% respectively. In contrast, the item that 
received the highest number of never responses was 
“encourage students to use their native language with their 
children”. The overall pattern of responses demonstrates 
great variability, with 18.4% of the respondents stating they 
never do so, 16.1% stating they rarely do so, 19.3% stating 
they sometimes do so, and 17% and 29.2% stating they 
usually and always use that practice. 

5.3.2. Convergent Validity 

Convergent validity describes the relationship between 
assessments [27]. If there is a positive correlation, the 
instruments can be seen to measure related constructs. The 
results of the Pearson-product moment correlations between 
the CRTS and the two sub-scales of the MCTS 
demonstrated predominantly significant and positive 
relationships. There were positive correlations between the 
scores of the CRTS and both the Teaching Skills (r = .587; 
p < .01) and Knowledge sub-scales (r = .506; p < .01) of the 
MCTS. A correlation coefficient matrix of these scores 
can be seen in Table 4. These significant correlations in the 
expected positive direction provide support for the 
convergent validity of the CRTS, by indicating that those 
who use or desire to use more culturally responsive 
teaching practices are also more skillful at and 
knowledgeable about teaching students of diverse 
backgrounds. 


Table 4. Correlations between CRTS and MCTS Sub-Scales 


Sub-Scales 

1 

2 

3 

1 .CRTS 

1 



2. MCTS Teaching Skills 

.587** 

1 


MCTS Knowledge 

.506** 

.678** 

1 


Note. “Significant at the p < 0.01 level. 


These significant correlations in the expected positive 
direction provide support for the convergent validity of the 
CRTS, by indicating that those who use or desire to use 
more culturally responsive teaching practices are also more 
skillful at teaching students of diverse backgrounds. 


6. Discussion 

The purpose of this investigation was to describe the 
culturally responsive teaching practices of adult ESOL 
teachers and to establish the psychometric properties of the 
Culturally Responsive Teaching Survey. These findings will 
lead to improved understanding of how adult educators 
incorporate culturally responsive teaching practices in 
ethnically and linguistically diverse learning environments, 
in addition to supporting the use of this instrument by 
researchers and practitioners. 

6.1. Patterns of Use 

These studies revealed a trend of regular to frequent use of 
the majority of the culturally responsive teaching practices 
indicated in the CRTS. The findings revealed that the 
sampled adult ESOL teachers respond to the ethnically and 
linguistically heterogeneous learning environment by 
reaching out and incorporating students’ learning styles and 
ways of knowing into their teaching instead of establishing 
classrooms which represent only mainstream American 
culture. There are unique characteristics of this learning 
environment that may contribute to this heightened level of 
culturally responsiveness. All adult English language 
learners enter the classroom with values and norms of their 
native cultural group. As a result, the need to position 
students’ cultures at the forefront of the learning process and 
utilize their values and experiences may be more compelling 
and obvious to ESOL teachers. 

However, there were some culturally responsive teaching 
practices that were used less frequently by adult English 
language educators. These teaching practices involved 
asking for student input about the learning process, practices 
which are grounded in learner autonomy and self-directed 
learning. They are also reflective of the element of 
developing attitude in the Motivational Framework of 
Culturally Responsive Teaching [1] and are practices posited 
to increase the learners’ positive regard for the content. 
Another area of limited use was the inclusion of lessons 
dedicated to discrimination. Promoting critical inquiry and 
addressing real-world issues are tenets of culturally 
responsive teaching. However, findings from both studies 
revealed that adult education ESOL teachers did not 
regularly include lessons about anti-immigrant 
discrimination or bias. It is necessary to develop a better 
understanding of the rationale behind the less frequent use of 
these practices to determine how they relate to the overall 
construct of culturally responsive pedagogy. 

6.2. Psychometric Properties of the CRTS 

6.2.1. Factor Structure 

Using the four elements of Ginsberg and Wlodkowski’s 
model [1], this researcher hypothesized a multi-dimensional 
construct of the CRTS. However, findings from the 
exploratory factor analysis conducted in Study 1 support a 
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uni-dimensional structure. An explanation may be found in 
the interconnected and holistic nature of the model’s design. 
The Motivational Framework of Culturally Responsive 
Teaching was designed to assist practitioners in instructional 
planning with self-reflection at its foundation. The four 
elements are described as interconnected parts of a “holistic 
and systemic” [1, p. 34] model of classroom practice. All 
elements work in partnership to create an environment 
conducive to learning for students of diverse backgrounds. 

Another explanation may be linked to the survey items’ 
lack of sensitivity to nuances involved in culturally 
responsive teaching. The survey includes 17 items which 
were developed and validated through online questionnaires 
with multiple choice questions. The lack of open-ended 
questions may have resulted in an overly restrictive 
validation process and could be improved upon by 
conducting live focus groups. Using these, and additional 
qualitative methods, would allow for probing of participants 
in order to gain a better understanding of this discrepancy 
between culturally responsive teaching theory and its 
practice in adult ESOL classrooms. 

6.2.2. Convergent Validity 

Convergent validity describes the relationship between 
assessments [27]. If there is a positive correlation, the 
instruments can be seen to measure related constructs. In 
Study 2, scores from the CRTS and the MCTS sub-scales 
were analyzed for correlations through Pearson-product 
moment correlations. The hypothesis of a positive 
relationship between the two areas examined by the MCTS, 
multicultural knowledge and multicultural teaching, and the 
CRTS guided the examination of data. 

The MCTS approaches multicultural teaching as a 
complex and ongoing activity. The theoretical framework of 
the MCTS is a multi-dimensional construct of: continual 
critical reflection, motivation to increase awareness of 
diversity, and the connection between educator beliefs and 
praxis. It has been used in a variety of educational settings 
with acceptable levels of internal consistency and validity. 
Therefore, the positive correlations found in Study 2 
demonstrate the usefulness of the CRTS to adult educators 
interested in exploring the cultural responsiveness of their 
teaching. 


The results of these studies provide support for the 
reliability and validity of the CRTS. Findings suggest that 
the CRTS is a reliable uni-dimensional measure, whose 
scores demonstrate convergent validity through positive 
correlation with multicultural teaching knowledge and skills. 
The CRTS provides a useful tool for researchers to expand 
understanding of adult ESOL teachers’ strategies to 
incorporate students’ cultural identities into the classroom in 
the presence of ethnic, racial, and linguistic diversity. 
Further research will yield additional information about 
general patterns of behavior and should include 
examinations of differences based on teacher demographics. 

7. Implications for Future Studies 

There are areas to consider as the CRTS is used in 
additional research studies. Based on this investigation, it is a 
reliable and valid self-assessment survey of culturally 
responsive teaching practices appropriate for adult English 
language classrooms. However, additional refinement and 
expansion of the current item pool would add to its 
usefulness in describing teacher praxis. The survey includes 
17 items which were developed and validated through online 
questionnaires. The use of online questionnaires in the item 
pool development stage could be improved upon by 
conducting live focus groups with adult ESOL teachers. For 
example, during the validation stage, two items related to the 
use of native language were deemed not relevant to the adult 
ESOL classroom. Additionally, two items of the same nature 
were combined to create one item. This resulted in the 
inclusion of only two survey items related to the use of the 
students’ native language. However, a tenet of culturally 
responsive teaching theory is the importance of 
incorporating and facilitating the development of students’ 
native languages in order to promote academic success. 
During a focus group, the researcher could probe participants 
in order to gain a better understanding of this discrepancy 
between culturally responsive teaching theory and its 
practice in adult ESOL classrooms. Additionally, 
participants could describe the rationale behind the 
frequency of usage. This information would greatly add to 
the understanding of the implementation of a culturally 
responsive teaching approach. 
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Appendix 


A. Survey of Culturally Responsive Teaching Practices 


Item # 

Item Prompt 

1 

I include lessons about the acculturation process. 

2 

Examine class materials for culturally appropriate images and themes 

3 

I ask students to compare their culture with American culture. 

4 

I make an effort to get to know my students' families and backgrounds. 

5 

I learn words in my students' native languages. 

6 

I use mixed-language and mixed-cultural pairings in group work. 

7 

I use peer tutors or student-led discussions. 

8 

I use surveys to find out about my students' classroom preferences. 

9 

I elicit students' experiences in pre-reading and pre-listening activities. 

10 

I encourage students to speak their native languages with their children. 

11 

I have students work independently, selecting their own learning activities. 

12 

I spend time outside of class learning about the cultures and languages of my students. 

13 

I include lessons about anti-immigrant discrimination or bias. 

14 

I supplement the curriculum with lessons about international events. 

15 

I ask for student input when planning lessons and activities. 

16 

I encourage students to use cross-cultural comparisons when analyzing material. 

17 

I provide rubrics and progress reports to students. 
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